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ABSTRACT 

^ . . This paper explores the influence of field 

independent-dependent cognitive, style on second language test 
performance, especially as it relates to performance on the 
integrative type of measure known as the cloze test. Approximately 
250 college students enrolled in a first semester Spanish course 
formed the sample group for this correlational study. Students were 
administered the Group Embedded Figures* Testoffield 
dependence-independence along with* several measures of linguistic, 
communicative, and integrative competence. The results showed student 
field independence to be related consistently in a positive albeit 
modest fashion to second language test performance. Most notable was 
the correlation between field independence and. cloze test performance 
(r» t 43, p. .001). The relationship was less marked on other measures 
such as final course grade • (r«. 21, p.. 001). This suggests there may 
be a cognitive style bias operating in conjunction with cloze test 
performance. That is, such measures may call forth cognitive 
restructuring abilities more readily available to more field 
independent individuals. In turn, it implies the need to use some 
caution when employing or interpreting cloze tests for placement or 
achievement purposes. (Author) 
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INTRODUCTION 

During the past decade scholars in the field of second language 
testing have directed an increasing amount of attention to the cloze test 
procedure as a measure of general second language proficiency • A verbal 
cloze test presents the reader with a prose passage which has had words 
deleted systematically from the text* The reader must then fill in the 
blanks with the appropriate vprds'. Research studies have shown the cloze 
test to correlate rather well with other measures of second language 
proficiency. This suggests that it is a valid and reliable procedure. 
However, since its first application to the L2 learner (Carroll, et al., 
1959), there has been considerable uncertainty about exactly which language 
skills and cognitive processes are tapped in cloze test performance. As a 
result there is continued controversy about the validity of this procedure 
as a test of general L2 proficiency (Alderson, 1979). , The basic question 
remains, is success on a cloze test solely a function of second language 
proficiency, or do other non-linguistic factors influence the ability to 
fill in the "blanks appropriately? In this paper we will explore the 
influence „ of one non-linguistic factor, field dependent-independent 
cognitive style (FD/I), on L2 cloze test performance by presenting further 
analyses of data collected during a study which is described elsewhere 
(H&nsen knd Stansfield, 1931, 1982). 

BACKGROUND 

As mentioned above, the verbal cloze test presents the reader with a 
prose passage in which words have been systematically eliminated from the 
text. The deletions usually occur at every fifth to tenth word while the 
first and last sentences of the passage are generally left intact. The 
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reader must then fill in the blanks with the appropriate words to complete 
the text. Scoring of the insertions varies since credit can be given for the 
exact word only, for synonyms, or for any semantically acceptable word choice. 

The procedure was pioneered by Taylor (1953) who experimented with it 
as a measure of contextual redundancy. Taylor derived the name from the 
concept of closure in Gestalt psychology.. Gestaltiste believe that learning 
follows a sequence through which one first 0 understands the whole, or broader 
issues, and then grasps the individual details. Similarly, the cloze 
procedure requires the student to perceive the whole, by filling in the 
missing words, as if they were not missing at all (Stansfield, 1980). 

Since Taylor's initial work, the cloze test h£s been used for a variety 
of purposes. In particular^ it is recognized as a reliable and valid measure 
of reading comprehension and text readability for native speakers of English 
(Alderson, 1979; Readance et al., 1980). When applied to non-native speakers, 
it is viewed by many as a valid and reliable measure of general second 
language proficiency (Bialystok and Howard, 1979; Aitken, 1977; Oiler, 1976). 
Proponents of the test suggest that it is an integrative test of global skills 
in the second, language (Oiler, 1976). As such it measures overall or general 
proficiency to a greater degree than do more traditional discrete-point tests 
of vocabulary and grammar. At present the cloze test is used as a testing 
device on standardized second language proficiency measures, such as the 
Secondary level English Proficiency Test, and on foreign language classroom 
tests at all levels. 

Research on the cloze procedure in the L2 setting has basically 
focused on the correlation between cloze test performance and scores on 
other types of second language tests such as dictation and reading 
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comprehension tests, and on standardized proficiency measures like the 
Test of English as a Foreign ^anguage (TOEFL). In a review of the 
literature Aitken (1977) reported that the^ -majority of studies show that 

cloze performance correlates well with other measures of L2-proficiency. 

For example, in an early study, Darnell (1968) obtained a correlation of 
•84 between cloze performance and scores on the TOEFL. Oiler (1972) found 
correlations of .75 and .83 between the- cloze and the UCLA English as a 
, Second Language Placement Exam. Canadian researchers report correlations 
ranging from .52 tc .70 between cloze data and second language achievement. 
(Swain, Lapkin and Barik, 1976; Lapkin and Swain, 1977). Given these 
fairly high and consistent correlations, proponents of the cloze procedure 
have argued that it offers an easily constructed, reliable, and valid test 
of general L2 proficiency. 

Yet some researchers in the field of second language testing urge 
caution before embracing those assumptions until more is known about the 
validity of this procedure to measure L2 proficiency. For instance, 
Alderson (1979) reports that the utility of cloze tests as actual measures 
of second language skill varies widely. He shows that performance differs 
as a function of text difficulty, scoring procedures, and word deletion 
frequency. It is his view that as those factors vary, the cloze measures 
different abilities. Thus its reliability and validity vary from one 
situation to another. 

One point of confusion arises frpm the fact that we have little 
understanding of the way or degree to which cloze testing actually taps 
or reflects second language processing. This dilemma is true even in 
respect to the well established use of the cloze test as an indicator of 
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o reading comprehension for native speakers (Readauce et al. , 1960; Bormuth, 

\ 

1969; Weaver, 1965). Bialystok and Howard (1979, p. 27) recognize this 

problem in the area of L2 testing also: 

However, in spite of the sample demonstration of cloze test 

reliability as giveiTbyTHe correlations- with-numerous^other^ 

proficiency measures, the precise skills measured by th> cloze 
test and the problem-solving processes which they presuppose 
have not been specified. 

Oiler and Conrad (1971) acknowledge this deficiency but pose the 
question: "Ie it necessary to know exactly what a test is a test of in 
order to make use of it?" (p. 187). Thfey proceed to respond to that 
question in the negative. Nevertheless, most psychologists an? specialists 
in educational measurement would affirm the need to establish the construct 
validity of any test. Construct validity in language testing must 
necessarily be based on a theory of language processing that bears a 
relationship to the processes called forth on the test. Psycholinguists 
suggest that both receptive and productive language processing involves a 
strategy of sampling, predicting, testing, and confirming meaning based 
on one's internalized language system (Goodman, 1971; Aitken, 1977). 
Similar processes appear .to be called forth in solving a cloze task. 

Theoretically, in a cloze test a person needs to employ a large 
number of the interrelated skills that comprfse a language system (e.g., 
lexical, grammatical, contextual) in order to predict accurately what word 
most appropriately fits into each empty space. This prediction is said 
to take place through an hypothesis-testing strategy based on" one's 
internalized language competence. According to Oiler (1973) the taker of 
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an L2 cloze test inters or projects an acceptable word on the basis of a 

whole or complete message. As one notices the details and samples from 

» 

the information available while trying to fill in the spaces, one 
formulates hypotheses about the information expefcted to follow* By 
, further sampling of subsequent information,, the original hypotheses are 



conf irned or challenged. If they~~are repudiated, - one-revises the -firsts ... 
expectations, restructuring information to form a new hypothesis • For the 
second language learner, the* accuracy of this strategy on a cloze test or 
a dictation reflects the degree of underlying, internalized 'second 
language competence* 

Bialytsok and Howard (1979), concerned with identifying the actual 
processes involved in solving- 1 verbal cloze tasks, investigated the skill 
of inferencing as a factor in cloze performance* They defined inferencing 
as the ability to exploit maximally all available information sources in 

order to arrive at new insights into unknown aspects of the second language « 
They hypothesized that if inferencing were involved in cloze solutions, 
then factors that 'facilitated' inferencing should enhance performance on a 
cloze test* In their study, cues and instructions to facilitate the use 
of • inferencing behavior did result in improved cloze test performance* 
They concluded that inferencing was an integral component in performance 
on cloze tests* ■ 

THE PROBLEM 

As outlined above, inferencing has been identified as an integral, 
nonlinguistic factor in L2 cloze test performance. Interestingly, the 
psychological literature describes the cognitive style construct 
of field dependence-independence as a cognitive factor that affects 
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hypothesis-testing, inferencing, and restructuring behavior on various 
problem-solving tasks (Goodenough, 1976; Witkin, Moore, Goodenough, and 
Cox, 1977). Thus, field dependence-independence might also be a "non- 1 
linguistic factor that influences L2 cloze test performance. 

Field dependence-independence refers to individual differences in 
preferred ways of perceiving, organizing, analyzing, or recalling 
information and experience. Fieldndependence- indicates- a tendency to rely 
on external frames of reference in cognitive activities and is thought to 
.fofcter 'skill in interpersonal relations, whereas field independence suggests 
reliance on internal rules or strategies for processing information and the 
existence of mental restructuring abilities (Witkin rnd Goodenough, 1977). 

* Witkin, Mocre, Goodenough, and Cox (1977) explain that persons with a' 
well articulated, field-independent cognitive style are apt to analyze 
actively the elements of a perceptual field when.it is organized and to impose 
structure on a field which lacks an inherent organization. Field-independent 
persons are likely to employ such mediatioaal processes or strategies as 
analyzing, structuring, hypothesis-testing, and inferencing to generate 
solutions to problems. They appear to experience the details of a "field** as 
separate elements and they can alter 'that field or context when necessary to' 
accomplish the task. Moreover, they ^ehave as though governed by general 
internalized principles which they have actively abstracted from their 
experiences. In contrast, field-dependent persons make less use of these 
mediational strategies in information processing. They are likely to use the 
"field" as they find it, to make less use of surrounding information, and to 
have more difficulty analyzing that information to solve a particular problem 
(Readance et al., 1980). In other words, they are not likely to exploit 
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maximally all information sources. This exploitation of information sources 

is the definition of inferences behavior offered earlier. However, it is 

• % * 

passible to train field-dependent persons to utilize an analytical • 
hypothesis-testing approach in appropriate situations (Witkin et al., 1977) • 

If the L2 cloze test is conceived as^ja^task-which-asks the test-taker 
to infer or predict the appropriate word in order to fill the gap through an 
hypothesis-testing strategy, it could be related to the cognitive restructuring 
abilities fostered by a field-independent cognitive style. As a result, the 
test may be making cognitive demands which allow the field-independent person 
to fill in the blr^ks more easily or accurately regardless of second .language 
proficiency.* Field-dependent persons, on the^ other hand, may be at a 
disadvantage when taking this type of test, since they aren't as likely to 
utilize the strategies helpful to the solution of L2 cloze problems. In that 
event a cognitive style bias would be operating in cloze performance. • • a 
bias which would lessen the validity of this instrument as a test of general 
second language .prof iciency. y 

In an attempt to address the issue of possible cognitive sty^fe'bias in 
the cloze procedure, this paper compares the achievement patterns of foreign 
language students on a variety of Spanish proficiency measures, including the 
cloze test, in order to ascertain the relationship between performance on the 
different tests and the degree of field dependence-independence. ^ 

METHOD 

Subjects . The subject? for the study were 293 college students in an 

introductory Spanish course at the University of Colorado. The 16-^week course 

> 

emphasized both linguistic and communicative competence through .large group 
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lectures (two hours per weeek), small group recitation classes (three hours 
per week), language laboratory sessions, and textbook-workbook excercises. 

Instrumentation . The "students' degree of field dependence-independence 
was determined by the Group, Embedded Figures Test (GEFT). The GEFT (Oltman, 
Raskin, and Witkin, 1971) is a group administered test which requires the 
subject to outline a simple geometric shape within a complex design. The 
subject must -locate" or separate the relevant information from the contextual 
field and restructure it to design the correct shape.' In theory, this task 
discriminates the extent to which the person perceives- analytically and is 
able to identify the relevant information within the organized field. , 

Foreign language proficiency was assessed in terms of three areas of 
competence: linguistic, communicative, and integrative. Linguistic 
competence was defined as the ability to use. basic structural units of 
Spanish. This, was tested by each student's Written Exam Grade Average, 
derived from scores on six unit tests designed to assess mastery of 

r 

grammar; and by scores $n the Final Exam, a comprehensive discrete-point 
achievement test similar to- the unit tests in format. Communicative 
competence, defined as the ability to give and receive oral messages in 
Spanish, was assessed by each student's Oral Grade Average, obtained from . 

'performance on oral tests of communicative ability given throughout the 
semester; and by teacher ratings on an Oral Skill Evaluation questionnaire. 
Integrative competence, interpreted as general langu ge proficiency or a 
combined linguistic and communicative- competence was measured via each 
students' Final Course (jrade and Cloze Test score. 

Procedures . Toward .the end of the semester the GEFT was administered ^ 

"^Tall^he students who were present at a iarge-group lecture session. 
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Everyone was urged to participate in the test, though a few students chose 
to study instead. No effort was made to coerce those students or to test 
anyone who was absent. As a -result. GEFT scored were obtained for 253 ~> 

•■ ■' 1- '! ■ ■ 

students. American College Test (ACT) English and 1 Math scores were 
recorded for a subset of 102 students on whom such data was available in 
the univei^Bity Admissions Office.* The course instructors provided* the 
Oral Skill Evaluation fot each student 4s well as the various exam and 0 
course grades. The Cloze Test was given at the. time -of the Final Exam. 
Data Analysis Procedures . A correlational design was chosen to . K . 
* analyze the relationship ^between student FD/I and Spanish achievement*. 
The initial procedure involved obtaining Hoyt reliability data on the GEFT- 



(.90) and the Cloze Test (.75). Pearson product-tojoment correlations were 

• *** 

then established between the several variables, cdrrecting for attenuation 
wherever possible. For a subgroup of 102 .students correlations were next 
obtained between academic aptitude, Spanish achievement,* and FD/I. In a 
further step academic aptitude was^removed from the- correlations -by a 

first-order partial correlation technique. 
* 

RESULTS AND DISCUSSION 

The correlations between the various measures of Spanish language 

proficiency ranged from .60 to .93. They are presented iji TaBle 1. These 

findings indicate that a substantial relationship exists between several 

» 

diverse types of language tests. This suggests that the various instruments 
are measuring overlapping language skills or a general aspect of language 
competence. It should biT noted, however, that the categories were generally 
lowest between the Cloze test and the other measures, ranging from .60 to 
.80. While these correlations are rather .strong p the amount of shared 
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Insert Table 1 about here 



variance between the Cloze Test and the other measures of language 
competence ic slightly less than the amount of variance those measures 
share with each other. Thus the Cloze Test appears to be tapping seme 
ability that is not incorporated into the other measures as 'completely as 
it is into the Cloze. 

• / 

* The correlations between GEFT score and the measures of Spanish 
proficiency were^all 'positive, but modest. Since a higher GEFT score 
indicates a greater degree of field independence, the positive nature of 
these correlations shows that a field-independent cognitive style is 
associated with a higher level of achievement on all measures of second 
language proficiency. For traditiorial measures, such as course grades and 
discrete-point grammar tests, the correlations with FD/I ranged from .20 
to .28. Yet the^ correlation between FD/I and Cloze Test score rose to 
• 43, a notable difference. 

When discussing the relation of cognitive style to scholastic 
achievement, academic aptitude should' be held constant. Although the 
evidence reveals that FD/I is a factor iri cognition, separate from general 

V * 

intelligence (Vernon, 1972), there is somfe overlap with both verbal and, 
quantitative' aptitude (Witkin, ModreJ Oljtman, Fiedman, and Owen, 1977). In 
this study, these constructs were 'assessed for a subgroup of students via 

A.C.T. Etjglish 4 and Math scores.' The correlation for verbal and quantative 

• * - V 

9 aptitude with FD/I was .32 and' .48 respectively, as shown in Table 2. The 



12 



correlations between academic aptitude and Spanish proficiency were in the 
0.I6 to .46 range. Verbal aptitude showed a somewhat stronger and more 

Insert Table 2 about here. 



consistent relationship to second language achievement than did mathematical 
aptitude, except on the Cloze Test. In that instance, A.C.T. Math score 
correlated .46 with the cloze measure while A.C.T. English scores exhibited 
a correlation of .39. Thus the more positive relationship was demonstrated 
between quantitative aptitude and cloze performance. 

It is noteworthy that GEFT and Cloze Test scores show a nearly identical 
pattern of correlation with A.CT. Scores and that both are more related to 
quantitative ability than to verbal aptitude. In addition, they both 
correlate more highly with mathematical aptitude than do the other Spanish 
achievement tests. Apparently the GEFT and the Cloze Test are tapping the 
same aptitude construct to a greater degree than_are_the-other— instruments^ — r 

^IIT^fd^F^to^disambiguate the overlapping relationships between academic 

aptitude, Spanish proficiency, and field dependent-independent cognitive 
style, the stronget of the two aptitude measures, the ACT Math score, was 
removed from the correlation through a partial correlation procedure. That 
analysis is presented in Table 3. The effect of removing quantitative 
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ability from the correlations between GEFT and the traditional measures of 

Spanish proficiency is to reduce the relationships to a non-significant 

lever (r * .07 to .IS). This is to be expected, sinfce when -we partial out 

Si- 
aptitude it should be impossible to predict achievement. t However, L2 Cloze 

Test 8cc>res continue to correlate~signlf i~C"antly-with~FD/I (r »—.22- j p <-.05)— 

when^aptitude is removed. This suggests that Cloze Test performance is 

c 

influenced to a greater degree by field independent cognitive style than are 
traditional measures of Spanish proficiency. 

Since the correlations between GEFT and the Cloze Test are much higher 
than those between GEFT and the other measures of Spanish proficiency, it 
seems that a cognitive style bi as mav be operational in cloze solutions. 
That is, the evidence indicates that field independent individuals do indeed 
fill in the blanks on a Cloze, Test more easily than do field dependent 
persons. Their FI cognitive restructuring abilities are more conducive to 
success on a cloze reconstruction task. Based on this data, it appears that- 
general second language proficiency and academic aptitude do not fully 
explain L2 Cloze Test performance. The cloze incorporates a non-linguistic, 
cognitive ntyle factor as well. It is noteworthy that Carroll, Carton and 
Wilds (1959, p. 116) obtained similar findings in an initial investigation 
of, the cloze done for the College Entrance Examination Board. After 
comparing the cloze with other measures they, concluded that it is "affected 
by various sources of extraneous variance," including certain intellectual 
traits. Further research into the actual cognitive processes involved it^ 
solving L2 cloze tests may lend insights into how field independence 
influences performance on cloze reconstruction tasks* 

*> • 
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Correlations Between Six Measures 
of Spanish Proficiency ^and Student 
Field Independence % 



t 
T 

T 

^s— o— ■ , 


• 
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Written 
Exam 
Grade 
Average 


Final 
Exam 


■ " i 

•v 

Oral 

Gvnrlt* 

- Average I 


Evaluation 


Final 

Course 

Grade 


Cloze 
Test 


* 


Final Exam 

Oral Grade . 
Average 


.70 


.67 




i 

! 
\ 








Oral Skill 
Evaluation 


.70 


.70 


.76 








! 


Final Course 
Grade 


.93 


.88 


.76 


.73 








Cloze Test 


.68 


.80 


.60 


.64 


.69 






GSFT 1 


♦24 


.28 


.20 


.21 


.21 


.43 




" "i 


s " * 


/ 












p < .001 in all 


instances. 




) 









X A higher GEF£ score indicates a relatively greater degree of field 
independence. 
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. „ Tabie"2 

Academic- Aptitude Correlations with Six Measures 
of -Spanish -Proficiency, and. .Field Independence 



'f 




ACT — English 


ACT — Math 


n 


\ Written Exam 

Grade Average * 




.-36*** 


.36*** 


102 


; — ■ Final- Exam 




.34*** 


.36*** 


102 


I Oral Grade 
i Average 




• .35*** 


.25** 


95 


Oral Skill 
Evaluation 




.31*** 


.16* 


102 


Final Course 
Grade 




.34*** 


.22* 


102 


? Cloze Test 




.39*** 


.46*** 


90 


GEFT 




.32*** 


..48*** 


102 



*p < .05 
**p < .01 
***p <.O01 
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Table 3 ~- 

Correlations Between Measures of Spanish Proficiency 
and Field Independence, 
Partialiing Out Academic Aptitude 
(ACT Math Score) 



GEFT n 

c Written Exam 

Grade Average .07 102 

« * 

Final Exam .13 102 

Oral Grade Average TOB 95 

Oral Skill Evaluation .15 102 

Final Course Grade ' .12 102 

Cloze Test .22* 90 



r -*p^<.05 . 

< — ' 
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NOTES 

1, 



Since tfie context of this study is a first semester foreign language 
course, we will use the terms achievement and proficiency interchangeably. 
For a detailed explanation of this usage of terminology see Stansfield (1981) 



* 
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